Search for: All records

Abstract Objective.The RSVP Keyboard is a non-implantable, event-related potential-based brain-computer interface (BCI) system designed to support communication access for people with severe speech and physical impairments. Here we introduce inquiry preview (IP), a new RSVP Keyboard interface incorporating switch input for users with some voluntary motor function, and describe its effects on typing performance and other outcomes.Approach.Four individuals with disabilities participated in the collaborative design of possible switch input applications for the RSVP Keyboard, leading to the development of IP and a method of fusing switch input with language model and electroencephalography (EEG) evidence for typing. Twenty-four participants without disabilities and one potential end user with incomplete locked-in syndrome took part in two experiments investigating the effects of IP and two modes of switch input on typing accuracy and speed during a copy-spelling task.Main results.For participants without disabilities, IP and switch input tended to worsen typing performance compared to the standard RSVP Keyboard condition, with more consistent effects across participants for speed than for accuracy. However, there was considerable variability, with some participants demonstrating improved typing performance and better user experience (UX) with IP and switch input. Typing performance for the potential end user was comparable to that of participants without disabilities. He typed most quickly and accurately with IP and switch input and gave favorable UX ratings to those conditions, but preferred standard RSVP Keyboard.Significance.IP is a novel multimodal interface for the RSVP Keyboard BCI, incorporating switch input as an additional control signal. Typing performance and UX and preference varied widely across participants, reinforcing the need for flexible, customizable BCI systems that can adapt to individual users. ClinicalTrials.gov Identifier: NCT04468919. 

In this paper we present a hybrid neural network augmented physics-based modeling (APBM) framework for Bayesian nonlinear latent space estimation. The proposed APBM strategy allows for model adaptation when new operation conditions come into play or the physics-based model is insufficient (or incomplete) to properly describe the latent phenomenon. One advantage of the APBMs and our estimation procedure is the capability of maintaining the physical interpretability of estimated states. Furthermore, we propose a constraint filtering approach to control the neural network contributions to the overall model. We also exploit assumed density filtering techniques and cubature integration rules to present a flexible estimation strategy that can easily deal with nonlinear models and high-dimensional latent spaces. Finally, we demonstrate the efficacy of our methodology by leveraging a target tracking scenario with nonlinear and incomplete measurement and acceleration models, respectively. 

The effectiveness of human-robot interactions critically depends on the success of computational efforts to emulate human inference of intent, anticipation of action, and coordination of movement. To this end, we developed two models that leverage a well described feature of human movement: Gaussian-shaped submovements in velocity profiles, to act as robotic surrogates for human inference and trajectory planning in a handover task. We evaluated both models based on how early in a handover movement the inference model can obtain accurate estimates of handover location and timing, and how similar model trajectories are to human receiver trajectories. Initial results using one participant dyad demonstrate that our inference model can accurately predict location and handover timing, while the trajectory planner can use these predictions to provide a human-like trajectory plan for the robot. This approach delivers promising performance while remaining grounded in physiologically meaningful Gaussian-shaped velocity profiles of human motion. 

State-of-the-art human-in-the-loop robot grasping is hugely suffered by Electromyography (EMG) inference robustness issues. As a workaround, researchers have been looking into integrating EMG with other signals, often in an ad hoc manner. In this paper, we are presenting a method for end-to-end training of a policy for human-in-the-loop robot grasping on real reaching trajectories. For this purpose we use Reinforcement Learning (RL) and Imitation Learning (IL) in DEXTRON (DEXTerity enviRONment), a stochastic simulation environment with real human trajectories that are augmented and selected using a Monte Carlo (MC) simulation method. We also offer a success model which once trained on the expert policy data and the RL policy roll-out transitions, can provide transparency to how the deep policy works and when it is probably going to fail.